Graphical Models: Structure Learning

INTRODUCTION The article GRAPHICAL MODELS: PARAMETER LEARNING discussed the learning of parameters for a xed graphical model. In this article, we discuss the simultaneous learning of parameters and structure. Real-world applications of such learning abound and can be found in (e.g.) the Proceedings of the Conference on Uncertainty in Arti cial Intelligence (1991 and after). An index to software for parameter and structure learning can be found at http://www.cs.berkeley.edu/ murphyk/Bayes/bnsoft.html. For simplicity, we concentrate on directed-acyclic graphical models (DAG models), but the basic principles described here can be applied more generally. We describe the Bayesian approach in detail and mention several common non-Bayesian approaches. We use the same notation as the article on parameter learning. In particular, we use X = (X1; : : : ; Xn) to denote the n variables that we are modeling, x to denote a con guration or observation of X, d = (x1; : : : ;x) to denote a random sample of N observations of X. In addition, we use Pai to denote the variables corresponding to the parents of Xi in a DAG model and pai to denote a con guration of those variables. Finally, we shall use the terms \model" and \structure" interchangeably. In particular, a DAG model (and hence its structure) is described by (1) its nodes and arcs, and (2) the distribution class of each of its local distributions p(xijpai).